Jereys' Prior for Layered Neural Networks

In this paper, Je reys' prior for a neural network is discussed in the framework of the Bayesian statistics. For a good performance of generalization, the regularization methods which reduce both cost function and regularization term are commonly used. In the Bayesian statistics, the regularization term can be naturally derived from prior distribution of parameters. Je reys' prior is known as a typical non-informative objective prior. In the case of neural networks, however, it is not easy to express Je reys' prior as a simple function of parameters. In this paper, numerical analysis of Je reys' prior for neural networks is given. The approximation of Je reys' prior is given from a parameter transformation getting to make Je reys' prior as a simple function. Some learning techniques are also discussed as applications of these results. keywords: Je reys' prior, bayes, regularization, neural network, generalization, weight decay, initial parameter